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Abstract 

We consider a scheduler for the downhnk of a wireless channel when 
only partial channel-state information is available at the scheduler. We 
characterize the network stability region and provide two throughput- 
optimal scheduling policies. We also derive a deterministic bound on the 
mean packet delay in the network. Finally, we provide a throughput- 
optimal policy for the network under QoS constraints when real-time and 
rate-guaranteed data traffic may be present. 

1 Introduction 

Scheduling has always been an indispensable part of resource allocation in wire- 
less networks. The seminal work of Tassiulas et al. {22, and later [53], [53] con- 
sidered the case where both channel states and queue lengths are fully available 
to the scheduler. It was shown that the MaxWeight algorithm, which serves the 
longest connected queue, is throughput-optimal. Subsequently, the MaxWeight 
algorithm was found to be throughput-optimal in many other settings as well 
([I]-[I1] and the references therein) using tools from Lyapunov optimization. 
Some other works (see [3S|, [53], [35]) also approach the scheduling problem us- 
ing convex optimization and dual decomposition techniques. 5 even considers 
the role of imperfect queue length information on network throughput, show- 
ing that the stability region does not reduce. But, in all these cases, accurate 
information about channel-state is assumed as a modeling simplification. 

In a real-life network, e.g., Long Term Evolution (LTE) [H] or IEEE 802. 16e 
WiMAX, the channel-state information fed back to the transmitter can have un- 
certainty. The primary reason being that although resource-allocation is done 
at the finer granularity of a Physical Resource Block (PRB), channel-state in- 
formation is still fed back at the coarser granularity of a subband, which is 
a group of PRBs. This is done to reduce the feedback traffic from the users 
to the Base Station (BS). However, this averaging causes information loss and 
hence, the resulting uncertainty at the scheduler. Moreover, uncertainty might 
be present in the channel-estimates because of the very process of estimation. 

Some recent works have, hence, tried to model this uncertainty in the channel- 
estimate. In [2, the authors show that infrequent channel-state measurement, 
unlike infrequent queue length measurement, reduces the maximum attainable 
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throughput. [13] considers the effect of inaccuracy of channel estimation on 
throughput, but does so assuming a specific probabihty distribution of the 
channel-state and does not study the stability of the data queues either. [T^ 
attempts at modeling channel- and queue-state uncertainty by considering the 
case where only heterogeneously delayed information is available at the sched- 
uler. They however assume knowledge of the channel-state transition probabil- 
ities. In [30], the authors study scheduling with rate adaptation in a single-hop 
network with a single channel under channel uncertainty. They consider cases 
when the channel estimates are inaccurate but complete or incomplete know- 
ledge of the channel-estimator joint statistics is available at the scheduler. The 
authors, however, assume that the channel-estimates are independent across the 
channels for each user. 

Delay performance of various wireless systems has also been investigated by 
many researchers recently. Among previous work in the area, the authors in |29j 
study the problem of opportunistic scheduling of a wireless channel while also 
trying to minimize the mean delay. Neely |19j has given a 0(1/(1 — p)) delay 
bound in the case of ON/OFF channels and a 0{N / [1 — p)) bound for multi-rate 
channels for the classical Max Weight algorithm for a network of size N and any 
traffic input-rate vector within a p-scaled version of the stability region (where 
< /f)< 1). Subsequent work in [TT] established 0(1/(1 - p)) and 0{N/{1 - p)) 
delay bounds for the case of single-hop and multi-hop networks, respectively, for 
ON/OFF channels and under both i.i.d. and Markov modulated arrival traffic 
scenarios. |16j derives lower and upper bounds on the delay in a wireless system 
with single-hop traffic and general interference constraints. 

Our contributions of this paper are as follows: 

• Firstly, we model the channel-estimate inaccuracy and characterize the 
network stability region. Compared to [5D] , we allow the channel estimates 
to have dependence among themselves, which is a more realistic situation 
in a modern LTE or WiMax network. Besides, we study a multi-channel 
setup whereas they consider a single channel. 

• Secondly, we propose two simple MaxWeight based scheduling schemes 
that achieve any rate in the interior of the stability region. 

• Thirdly, we derive an 0{N/ {1 — p)) delay bound for our system under one 
of the throughput-optimal policies we propose. 

• Lastly, we propose a throughput-optimal policy for the network under 
traffic with heterogeneous Quality of Service (QoS) constraints and present 
some numerical results studying its performance. 

The remainder of the paper is organized as follows. In Section II, we de- 
scribe the system model and the assumptions made on the arrival and channel 
processes. Section III provides the network stability region. We also discuss 
an example that illustrates that partial channel information may lead to a loss 
of throughput. In Section IV, we propose two throughput-optimal policies for 
the network and prove their optimality. We also present some simulation re- 
sults in this section studying their performance. Section V provides a bound on 
the mean packet delay in the network. Section VI gives a throughput-optimal 
policy for the network under QoS constraints and studies its performance. We 
conclude with Section VII. 
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2 System Model 



Consider a multi-user cellular downlink system with N users and M orthogonal 
channels. The system operates with fixed-size data packets and in synchronized 
time slots denoted by i € {0, 1,2,.. .}. It may, for example, be a cellular down- 
link Orthogonal Frequency-Division Multiple Access (OFDMA) system. Each 
user has a separate queue for its data and Qi{t) denotes the queue- length (in 
terms of packets) for user i in slot t where i G {1, . . . , N}. We assume an infi- 
nite buffer at each queue. Ai{t) denotes the number of exogenous packet arrivals 
for user i in slot t. It is assumed that {Ai{t)} is i.i.d. from slot to slot with 
E{^i(0)} = Ai and with E{Ai(0)^} < oo for all i. However, in a particular slot 
t, Ai{t) may be dependent among themselves. 

The channel-state for user i and channel j in slot t is denoted by Xij(t). 
We assume that Xij{t) is i.i.d. from slot to slot and independent across users. 
Such an assumption holds, for example, in a wireless system like LTE where 
our channel corresponds to a PRE in the LTE system. A PRE has 180 kHz 
bandwidth which is close to the coherence bandwidth of the channel for a typical 
delay spread of 4-5 /iS (Sec. 5.3.2 in [21]). We define channel-state as the 
maximum number of packets that can be sent over the channel successfully 
without suffering an outage. We assume that Xij{t) G X where A" is a discrete 
state-space and ^{Xfj{t)} < oo. The scheduler has access to only estimates 
Sij{t) of the channel-state in slot t where i G {1, . . . , N} and j £ {1, . . . , M}. 
These estimates are used by the scheduler to schedule different channels to the 
users in slot t. The estimates for a particular user may be dependent on each 
other. This can be used to model the effect of averaging (or even calculating 
any deterministic function, for that matter) of the channel-gains as done in LTE 
systems which are sent by the users to the ES to reduce the feedback traffic. 
The only constraint we impose on Sij{t) is that Sij{t) G 5, where 5 is a discrete 
set, and that it is i.i.d. from slot to slot. As a shorthand, we shall use Q{t), A{t) 
and S(<) to denote the queue-length vector, arrival vector and channel-estimate 
matrix, respectively, at slot t. We use Ps(') to denote the probability mass 
function of the random variable S. We also assume that the channel/estimator 
statistics given by the set of probabilities F(^ij = a; | S = s), Vx G A" and s G 5 
is available at the scheduler. This can be achieved, possibly, using a mechanism 
that learns the statistics on-the-fly. We assume that a channel can be allocated 
to at most one user in a particular slot. We use the notation Ij{t) and Rj{t), 
j G {1, . . . , Af } to denote the user scheduled on channel j and the corresponding 
rate allocated to it, respectively, in the slot t. 

We can then write the queue evolution equation as: 

Q,{t + I) ^ iQ^it) - n,(t))+ + A,{t) (1) 
where a+ = max{a, 0} and 

M 

In this equation, we have assumed that the packets sent on channel j are received 
successfully if and only if Rj{t) ^ Xij{t), i.e., probability of error is negligibl^ 

^To be precise, we can transmit data at rate Xij (t) with any arbitrarily small probability of 
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if i transmits at a rate less than or equal to Xij{t) on channel j. Under these 
conditions, {Q{t)} is a countable state Markov chain. For simplicity, we will 
assume it to be irreducible. But the general case can be easily handled. 

We will use the following notation: C[A\ is the convex hull and mt[A] is 
the interior of set A, Ij is the i*^ coordinate vector and and 1 denote A''- 
dimensional vectors of zeroes and ones, respectively. 

Let L{Q{t)) be a Lyapunov function. We define one-slot conditional Lya- 
punov drift as 

A(Q(t)) 4 E{L(Q(t + 1)) - L{Q{t)) \ Q(t)}. (3) 

In the following, we provide an upper bound on A(Q(t)) which will be used 

later on. 

Lemma 1. For the quadratic Lyapunov function L{Q{t)) = "^f-i Qfit), 

N 

A{Q{t)) < E{U{t) I Q{t)} + 2 J2 Q^{t){^^ - I Q{t)}) (4) 

i=l 

where 

N 

u{t)^j2(^Ut)+f^Ut))- (5) 

Further, if E{A-f{t)} < oo and M{Xfj{t)} < oo for each i and j, there exists 
B < 00 such that 

N 

A{Q{t))^B + 2J2Q^{t){^i-H^^^{t) I Qit)})- (6) 

3 The Network Stability Region 

We first define the notion of stability we use in the paper. 
Definition 1. A queue Qi{t) is strongly stable if 

1 

limsup - yjlE{(5i(T)} < 00. 

t->cx> t 

r=0 

The network of queues is strongly stable if each individual queue is strongly 
stable. 

Strong stability implies positive recurrence of the Markov chain {(5(f) }. In 
general, it is a stronger notion than positive recurrence. Throughout the paper, 
we shall use the term "stability" to refer to strong stability. 

We characterize the network stability region of the system now. Consider the 
set of stationary policies G that base their scheduling decisions at time t only on 
(Q(t), S{t)) and the channel-estimator statistics. The network stability region 
is defined to be the closure of the arrival rates that can be stably supported by 
the policies in G. 

error (provided Xij (t) is less than the Shannon capacity) assuming the physical-layer coding 
scheme supports it. For example, we can use an appropriate Turbo code or LDPC code for 
our purpose. 
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Theorem 1. The network stability region A is given by 

M 



A = ^Ps(s)C 



se5 



where 



0, J2 nXrj > r*. (s) I S = s) r*. (s) 1,; V * £ {1, . . . , iV} 
r* (s) = argmax { P(Xy ^ a; | S = s) x}, 



ties 6em(; broken lexicographically. 

Proof The proof goes along the lines of the proof of Proposition 1 in 
We have included it in Appendix A for the sake of completeness. 



3.1 An example 

We illustrate the loss in throughput caused by partial channel information using 
a simple example. We show that scheduling schemes that naively trust the 
channel-estimate fed back as the true channel-state may perform much worse 
than the policies that don't. 

Consider a system with 1 user and 2 channels. The 2 channel are assumed 
independent. Also, P{Xii{t) = 0} = 0.5 = P{Xii{t) = 2} and P{Xi2{t) = 
0} = V{Xi2{t) = 6} = 0.5. Suppose we can only observe the arithmetic average 
of the two channel states and not the individual states. So, F{Sii{t) = s} = 
0.25 = F{Si2{t) = s}, for s e {0, 1,3,4}. Now, if we take the S value to be 
the true channel-state, it can be easily shown that the mean service provided 
will be|x(0-|-l-|-3-|-4)=2 packets per slot. However, due to the special 
choice of the support set, the S values give us complete information about the 
channel-state of both the channels. Then, it is easy to see that we can provide 
mean service of-| x (0-1-2-1-6-1-8) = 4 packets per slot. 

We note that a careful scheduling decision can even double the mean service 
rate as shown in the example. Though the example may appear a bit contrived, 
numerical studies in the next section show that performance gains due to clever 
scheduling may indeed be substantial in many realistic situations. 



4 Throughput-optimal poHcies 

In this section, we describe two throughput-optimal policies and also prove their 
optimality. Even though the STAT policy described in the proof of Theorem 
1 is throughput-optimal, we require knowledge of the arrival rates A for STAT 
to be able to perform the channel-allocation. The throughput-optimal policies 
described here, in contrast, just require the arrival rate vector to lie within the 
stability region (without knowing A) and need only knowledge of the current 
queue- lengths. This will be available to a downlink scheduler used at a BS. 
Both, of course, require knowledge of the channel-estimator statistics. Moreover, 
as will be seen later in the section, scheduling schemes that naively trust the 
channel-estimate fed back perform worse than the policies in this section. For 
notational simplicity, we shall drop all the slot indices in this section. Lyapunov 
drift analysis techniques are used to prove the throughput-optimality of the 
policies in this section. 
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4.1 Max Weight policy 



We consider the MaxWeight (MW) type policy described below. 

At each slot t, the channel-estimate S = s is observed, and the decisions Ij 
and Rj are computed separately, for each j, as follows: 

1. To each user z, assign rate Rij such that, 

Rij — argmax { P(Xy ^ x \ S = s) x}. 

x<£X 

2. Schedule the user Ij that maximizes the rate-backlog-success-probability 
product: 

I J = argmax {Q, P(Xy ^ i?„ | S = s) R,^}. 

For the sake of completeness, we assume that all ties here are broken lexico- 
graphically. 

Theorem 2. The MaxWeight policy is throughput- optimal. 
Proof See Appendix B. 

4.2 Iterative MaxWeight pohcy 

We now analyses an iterative version of the above MaxWeight policy. These 
policies have also been studied in [3] and [4]. The new policy will be referred 
to as iMW. We study this policy because we find that it can give a lower mean 
delay than MW in some networks. In the iMW policy, we allocate the channels 
sequentially from 1 to M in M rounds taking into account the channels allocated 
so far. The virtual queue-lengths at the beginning of round j are considered for 
the allocation in round j. To aid the analysis, we use Qj-"'^ to denote the virtual 
queue-length of queue i at the beginning of round j of the allocation. Q.^^'' is 
defined to be Qi. We also assume that the set X contains a largest element 
denoted by Xmax- We can formulate the iMW policy now as follows. 

At each slot t, the channel- estimate S = s is observed. The decisions Ij and 
Rj are computed sequentially from channel 1 to M, as follows: 

1. Start with j = 1. 

2. For each j, do the following: 

(a) To each user i, assign rate Rij such that, 

Rij = argmax { P(Xij ^ x \ S = s) x}. 
xex 

(b) Schedule the user Ij that maximizes the rate-backlog-success-probability 
product: 

Ij = argmax {Q^ P(X„- > i?y | S = s) 

3. If j = M, stop. 

Else, put Qp^^^ (Q.p^ - l{Ij = i)Rj)~^, for l^i^ N, increment j by 
1 and continue. 
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Here, as before, ties are broken lexicographically. 
Theorem 3. The iterative MaxWeight policy is throughput- optimal. 
Proof See Appendix C. 



4.3 Simulations 

We present numerical simulation results to compare MW with iMW and also 
show the advantage of using channel estimators instead of the average channel 
gains. Firstly, we consider an ON/OFF system with P{Xij{t) = 0} = 1/2 = 
P{Xij{t) = 1}, for alH G {1, . . . , 10} and j G {1, . . . , 6}. We assume that each 
user estimates its channel-state correctly all the time but feeds back only the 
sum of the six channel-estimates to the scheduler. We do this to study the effect 
of averaging the channel-estimates as in LTeJ^ on the stability region. The naive 
scheduling schemes (MaxWeight [5^ and SSG 3 ) calculate the average channel- 
estimate from the sum fed back, round it down to nearest integer and use that 
for scheduling, taking it to be the true channel-state for each of the six channels. 
We consider symmetric Binomial(10, A) arrivals with equal rates for all users. 
We have simulated the system for 10^ slots for values of A from 0.01 to 0.5. The 
resulting simulated queue backlogs are shown in Fig. [T] We see a huge gain 
in the stability region compared to the naive algorithms. Similar results are 
obtained when the channel-estimates are rounded up to nearest integer instead 
of down. 
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Figure 1: Simulation for the ON/OFF system with symmetric traffic 



In the second set of simulations, we consider a multi-rate system with ¥{Xij{t) = 
x} = 1/4, for alH e {1, . . . , 10}, j G {1, . . . ,6} and x G {0, 1,2,3}. Here too, we 
study the averaging effect by making the naive scheduling schemes (MaxWeight 
and SSG) calculate the average channel-estimate from the sum fed back, round 

^We note that this is somewhat different from an LTE setup. Arithmetic mean or EESM 
| 21| is usually used in that case. 
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Figure 2: Simulation for the multi-rate system with symmetric traffic. The 
inset shows the queue backlog performance of iMW versus MW under moderate 
traffic conditions. 

it up to the nearest integer and use that for scheduling. We simulate the system 
for 10^ slots with symmetric Binomial(10, A) arrivals for values of A from 0.05 
to 1. Fig. [2] shows the resulting mean queue backlogs. We again see a decrease 
in the stability region when we use the naive schedulers. In this figure, we 
have also expanded the graph to show that iMW performs better than MW at 
least under moderate traffic conditions. Similar results are obtained when the 
channel-estimates are rounded down instead of up. 

We also simulated asymmetric systems and a similar behavior of the corre- 
sponding stability regions was observed. The results are not reported here for 
lack of space. 

5 A delay bound 

In this section, we derive an upper bound on the mean delay for the MW policy. 

Theorem 4. Assume A e int[A]. Then, the average delay in the system, 
denoted by D, satisfies the following hound: 



(7) 



where 



a = min 




(8) 



and < p < 1 and K < oo. 



Proof See Appendix D. 
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Note that the delay bound derived is OiNHl — p)) since ^jf'" „, / [[ — 
0(1), and K and /i are fixed constants. 

6 QoS constraints 

In the previous sections, there were no QoS constraints on the data. We only 
had to ensure that all the queues were stable. In this section, we consider three 
types of traffic: 



Real-time (RT) traffic: Demands an upper bound on packet dropping ratio 
and delay deadline guarantees. 

Rate-guarantee (RG) traffic: Demands minimum rate guarantees. 
• Best-effort (BE) traffic: Demands only queue stability. 

Let 7?., Q and B denote the set of RT, RG and BE users, respectively, in 
the system. The set of all users is denoted by N . We have a slotted system as 
before. However, the slots are grouped into frames and each frame consists of 
T consecutive slots. We assume that the RT packets have a deadline of T slots. 
These RT packets might be coming, for example, from a voice or a video source. 
RG packets coming into the system only require guarantees on minimum rate. 
These packets might be coming from a source with flow-control, such as by TCP 
protocol. Thus, the RG users may be treated, for our purposes, as packet sources 
with infinite backlogs. The minimum-rate guarantee sought by a RG user g is 
denoted by fig. The channel model is the same as before except that we assume 
Xbj{t) ^ Xmax < oo, V5 e S and for each channel j. All exogenous packets 
arriving into the system arrive only at the start of the frame and An [k] denotes 
the number of packet arrivals in frame k, k G {0, 1,2,.. .} for user n S Af. As 
before, {An [A;]} is i.i.d. from frame to frame with E{A„[0]} = A„ and with 
E{A„[0]^} < oo for all n G Af. In case, some of the RT packets arriving in a 
frame could not be served by the end of the frame, they are simply dropped. 
We denote the packet dropping ratio for user r by a,-, i.e., at most ar fraction 
of the packets arriving for user r can be dropped in the long term. 

In order to satisfy the QoS constraints of the RT and RG users, we use 
the concept of virtual queues (see |15| . [27] ) that evolve from frame to frame. 
Corresponding to each RT user r and RG user g, we have virtual queue-length 
processes {^[^]}fc°,o {-^sl^llfc^O' respectively. We can then write the queue 
evolution equations for the RT users as, 

r,.[fc+l] = iYr[k] ~ fir[k] + Ar[k]{l - ar))+ .yr e n, (9) 

where ^^[0] — 0. Pr[k] = Y^t'^kT ^ l^r{t), and Hr{t) is as defined in Simi- 
larly, the virtual queues of the RG users are updated as follows: 

Zg[k+l] = {Zg[k]-pg[k]+l3g)+ (lO) 

yg & G, where Zg[0] = and fJ,g[k] is defined just like fj,r[k]. Unlike the RT 
and RG users, the queues of the BE users evolve from slot to slot. Besides, 
BE users only maintain real queues since they do not have any QoS constraints 
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whatsoever. Qb{t) denotes the queue- length of BE user b at slot t and its 
dynamics are governed by the equations, 



Qb{kT + 1) = {Qb{kT) - iib{kT) + 
Qb(kT + t + 1) = {QbikT + t)- Hb{kT + t))+, 



(11a) 
(lib) 



Vt e {1, 2, . . . , T - 1} in frame k. We also define: 




if n e 7^, 

if n G Q, and 

if n e B. 



(12) 



6.1 Throughput-optimal policy 

We consider a modified version of the MaxWeight type policy described in the 
previous sections, which we call QoS-Max Weight (QMW). 

At each slot t in frame k, the channel-estimate S{t) = s is observed, and the 
decisions Ij{t) and Rj{t) are computed separately for each channel as follows: 

1. To each user n and channel j, assign rate Rnj such that. 



2. Schedule the user Ij(t), on channel j, that maximizes the rate-backlog- 
success-probability product below: 



For the sake of completeness, we assume that all ties here are broken lexico- 
graphically. 

Theorem 5. The QoS-MaxWeight policy is throughput- optimal. 
Proof See Appendix E. 

6.2 Simulation results 

We investigate the effect of partial information on the network stability region 
using numerical simulation. We consider a multi-rate system with P{Xij{t) — 
x} = 1/4, for ah i G {1,...,15}, j G {1,...,6} and x G {0,1,2,3}. There 
are 2 RT users (with the maximum dropping ratios and deadlines being 0.01 
and 10 slots for user 1, and 0.02 and 10 slots for user 2), 3 RG users (with 
minimum rate guarantees being 5, 2 and 1 packets-per- frame for users 3, 4 
and 5, respectively) and the rest 10 are BE users. RT users have Binomial(10, 
2.75) arrivals. RG users have Binomial arrivals with parameters 10 and the 
corresponding minimum rate. We have simulated the system for 5 x 10^ slots 
with symmetric Binomial(10, A) arrivals for the BE users with the value of A 
varying from 0.18 to 10. Fig. |2] shows the resulting mean sum queue backlogs. 



Rnj = argmax { P(X,y(t) ^ x\S{t) = s) x}. 



argmax{$„[fc]P(X„j(i) ^ i?„j |S(t) = s)i?,y }. 
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Figure 3: Simulation for the multi-rate system with partial channel information 
and with RT, RG and BE traffic. 

7 Conclusions 

We have considered a multi-rate wireless downlink with multiple channels and 
users. The base station schedules the traffic based on partial channel informa- 
tion. We have obtained its network stability region. We have then proposed two 
throughput-optimal schemes to achieve the stability region and proved their op- 
timality. We have also derived a bound on the mean packet delay in the network. 
Finally, we have proposed a throughput-optimal policy for the network under 
QoS constraints. A natural extension of our work would be to investigate the 
effect of uncertainty in the queue-length information. Also, more complicated 
traffic models are left to be studied in greater detail. 

Appendix A 

Proof of Theorem 1: 

Sufficiency: We show that A G mt[A] is a sufficient condition for stability. 
Now, since A is convex, for each s G S, there exists a scaling vector 7** and a 
scalar e > such that 

M 

A. + e < E E ^ 4 I S = 4 (s) (13) 

ses j=i 

for any user i, and where X^iLi 7f = 1; ^ '5- 

Consider the following stationary randomized policy, henceforth referred to 
as STAT: for channel estimate S{t) = s, allocate all channels to user i with 
probability 7^ and set the rate allocated on channel j to it to be (s). In that 
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case, the service rate of user i, call it Ui, is given by 

u,=E{^l,{t)} 

= E{fi,{t) I Q{t)} 

M 

= Ps(s)^P(X,, ^ r^(s) I S = s) 4(s). (14) 

se5 j=i 

Considering the quadratic Lyapunov function defined before and using (|13p 
and (jl4l) in ©, we can write the Lyapunov drift inequality as 

N 

A{Q{t))<B-2eJ2Q^{t)■ (15) 

i=i 

Now, we note that Q{t) evolves as a Markov chain with L{Q{t)) bounded below 
by zero and 'K{L{Q{t + 1)) | Q{t)} < oo. Also, notice that the conditional drift 
in ([T5|) is negative outside the finite set {Q{t) : Qii^) ^ j^}- Thus, using 

Theorem 2 from [18], we can say that the queue-length process is stable. 

Necessity: Here we show that A G A is a necessary condition for stability. 
For if not so, i.e., if A ^ A, we have a vector a and a scalar 6 > 0, such that for 
any v G A, we have 

N 

^ai{Xi - Vi) ^ 5, 

i=l 

from the Strict Separation Theorem (see, for example. Proposition B.14 in ^13,). 
Let us define the linear Lyapunov function L{Q{t)) = X^ili ctiQi{i)- Now, using 
definition ([3]) and the queue evolution equation ([T]), for any stationary policy in 
G, we can write 

JV 

A(g(i)) ^ ^a,E{A,(t) - ^l,{t) I Q{t)} 

i=l 
N 

-5]«,(A.-E{M.(i) I Q(t)}). (16) 

i=l 

Let Ui = K{iii{t) I Q{t)}. We now show that tt G A. 

E[^i,it) I Qit)] 

= 5]Ps(s)E[Ai,(i) I Q{ty,S{t)=s] 

sGS 

(M 
j2nRj{t) < x,,{t) I s{t) = s)i(/,(t) - z)i?,(t) 

(A/ 
5^P(4(s) < X,,it) I S(0 = s)l{I,{t) = z)4(s) 
i=i 

The second equality holds since the policy decisions Ij{t) and Rj{t) are com- 
pletely determined by Q{t) and S{t) within the class of stationary policies G. 
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The inequality at the end holds because of the way r*j{s) is defined. Thus, 

u e A. 

Thus, from the Strict Separation Theorem and ((T5| , we get 



N 
N 

It can be shown that E{\L{Q{t + 1)) - L{Q{t))\\Q{t)} < oo. Taking the finite 
set as {0} and noting that L{Q{t)) > for at least some Q{t), we see that 
the DTMC will not be positive recurrent, and hence, not strongly stable either. 
Thus, we conclude that the network will be unstable. 

Appendix B 

Proof of Theorem 2: 

Assume that we are working with A £ int[A]. Let the MW policy make 
decisions Ij and Rj, and the STAT policy make decisions /j and R'j. We then 
have, 

g, = i) Rj P(Xy ^ Rj I S = s) ^ Q,; = i) R] P(X,, ^ i?;. I S = s) 

Vi e {1, . . . , and Vj' e {1, . . . , A/}, since MW^ maximizes the expression in 
the LHS over the class G. Thus, summing over all i and j, we get 

N M 

^ ^ = i) Rj P(Xy ^ i?, I S = s) 

i=i i=i 

^ ^ ^ Q,: = i? ■ P(X», ^ I S - s). 

i=l j=l 

Therefore, 

AT M 

^ ^ g, E{ = i) R, l{X,j ^ R,) I Q; S = s} 

N M 

where we have used the fact that the policy decisions are completely determined 
by Q and S within the class G. Taking expectation w.r.t. S, 

N M 

i=i j=i 

N M 

^ ^ ^ E{i(/; = {} i?; ^ i?;.) i q}. (i?) 
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Note that STAT stabilizes the system since A G int[A]. Thus, for some e > 0, 
we can write 

M 

M 

where the equahty holds because STAT makes its decisions independent of 
the queue-lengths. Multiplying Qi on both sides and summing over all i 6 
{l,...,iV}, weget 

N N M 

^ g,(A, + e) < ^ ^ QMHij = i)Rjim ^ X,,) I Q} 

i—l i—1 j—1 

N M 

^ 5Z 51 Q^^iMIj = i)RAiRi ^ X,,) I Q} (18) 

i=l j=l 

where we have used (|17l) in the second inequality. Therefore, 

N / M \ N 

Y.qA^^ - = < \Q]]< -eE^* (19) 

i=i \ j=i / i=i 

Finally, using ((T9| in the drift inequality ([6]), we get 

AT 

i=l 

Now the queue evolves as a Markov chain since the scheduling and rate allocation 
decisions are taken based on the current queue-lengths and channel-estimate. 
The drift inequality above gives negative drift but for a finite set of queue- 
lengths. Hence, using Theorem 2 from [TSj, the network is stable. 



Appendix C 

Proof of Theorem 3: 

Assume that we are working with A G int[A]. Let the iMW policy make 
decisions Ij and Rj, the MW policy make decisions /j and R'j and the STAT 
policy make decisions /j' and Rj. For ease of exposition, we use Rij and R'^j as 
the intermediate allocation variables for iMW and MW policies, respectively. 
We then have, 

Qi^ Ri^, P(X,, ^ Ri^j I S = s) 
^ q\]-^''> Ri^j F{X,j ^ Ri^j I S = s) 
^Q^"'^ R'j,. P(X„- ^i?;,, I S = s) 
^ [Qr. - Mxmax) R'rj P(^^j ^ iJ^., | S = s) 
^ Qj, R'j.^ P(Xy ^ i?;,^. I S = s) - Mx^,,. 
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In the first inequality, we have used the fact that the virtual queue-lengths 
cannot increase with allocation rounds. The second inequality is due to the 
way iMW does the allocation at each round. Summing both sides over all 
j £ {!,..., M}, we get 

M 

Y^Qi, Ri,j P{x,,;^Rj^, I S = s) 

M 

Therefore, 

N M 

i=l j = l 
N M 

^ E E = i) nx^, > I S = s) - M^xl^,, 

2 = 1 3 = 1 

where we have used the property of indicator functions. Now, using the fact 
that — i) Rij — — i) Rj, for all i for the iMW policy (and a similar 
thing for MW), and proceeding in a way similar to the previous proof, we will 
get 

N M 

^^Q. E{i(/; = i) i?; i{x,, > i?;) i q} 

i=l 3 = 1 
N M 

^T.T.Q^ E{1(/, - t) R, ^ R,) I Q} + M^xL. (20) 

i=i j=i 

Now, using a STAT policy, as in the previous proof, to stabilize A + el e mi[A] 
for some e > 0, and using the property of MW for the expression obtained, as 
done in (fT5|). we get 

N N M 

^Q,(A, + e) <M2xL. + EEQ'^{1(^^^*)^^1(^^^^'^) I 

i=l i=l j=l 

where we have used the inequality in (j20p . Finally, rearranging the terms in the 
above inequality and using it in we get 

JV 

A{Q{t)) <{B + 2M'xl,,) - 2e ^ Q,{t) 

1=1 

Now, as before, we know that the queue evolves as a Markov chain. As before, 
observe that the drift is negative but for a finite set of queue-lengths. Hence, 
the network is stable. 
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Appendix D 

Proof of Theorem 4: 

Here too we drop the slot indices for the sake of brevity. Let the MW pohcy 
make decisions Ij and Rj, and any other pohcy G' in class G make decisions /j 
and R'y Then, we can write 

N M 

^^Q. E{1(/, - i) R, 1{X,, ^ R,) I Q} 

i=i j=i 

N M 

^ 5] ^ g, E{i(/j = z) Rr ^ Rr) \ q}. (21) 

i=i i=i 

Since A G mi [A], we can say that A S pA for some < p < 1. Consequently, 
— e A. Furthermore, defining as in (|5]), we can say that ^'1 S A where 
jj! — jf. Thus, owing to the convexity of A, we have 

A+ (1 -p)/i'l e A. 

Hence, we can find a STAT pohcy such that 

^ 1^ = ^^^'^ ^ i?;-) I q| = A. + (1 - p)/i'. (22) 
Substituting (|22|) into ((2T|) . and then using (|4]), we get 

A(g(t)) < E{C/(t) 1 Q(t)} - 2^JL'{l ^p)Y, Q^{t)■ (23) 
Finally, using Lemma 4.1 of [151 and the previous inequality, we can write 

t-l N t-1 

limsup-5]^E{g,(r)} ^— ^ limsup - J] E{C/(r)}. (24) 

*^o^ 2^(1 -p) t^^ 

Now, we notice that since the system evolves as an ergodic Markov chain 
with countable state-space, the time-averages are well defined, and hence, lim 
can be used instead of lim sup. Also, note that since E{Xfj{t)} < oo and 
TV is finite, we can write E{fif{t)} < KE{Af{t)} for ah i and K < oo. So, 
E{C/(t)} < {l + K) E{A2(t)} < oo. Using this fact and Little's Theorem, 
we thus get the delay bound in 

Appendix E 

Proof of Theorem 5: 

Squaring both sides of ([9]) and proceeding as in Appendix F, for each RT 
user r, we get 

Yr[k+1]^ -Yr[k]^ {Ar[k]{l-ar)f+^ir[k]^+2Yr[k]{Ar[lt]{l-ar)-^ir[k]). (25) 
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Similarly, for each RG user g, from ([TU)) . we get 

Zg[k + - Zg[k]' < + ^^[fc]2 + 2Zg[k]{f3, ~~ f,g[k]). (26) 

Squaring both sides of pT|) . for each BE user &, we get 

QbikT + 1)2 - QbikTf ^ Ab[k]^ + (ibikTf + 2QbikT){Ab[k] - A*b(fcT)), (27) 

Qb{kT + t+lf - Qb{kT + tf s$ ^ib{kT + t)^ - 2Qb{kT + i)Aifc(fcr + ^), (28) 
for t = {1, . . . , T — 1}. Now, observe that 

Qb{kT + t) ^ Qb{kT)~tfi 

max 5 

where a finite fj^max exists since we assumed that Xbj(t) ^ Xmax < oo, V6 G ;B 
and for each channel j. So, we can write 

-Qb{kT + t)nbikT + t)^ -Qb{kT)^ib{kT + t)+ ifx^r^axfJ-bikT + t) 
^ -QbikT)^ibikT + t)+t^il^,. 

Using the above fact, we can rewrite ([25]) as 

QbikT + t + lf - QbikT + tf ^ibikT + tf + 2t^^^^ - 2QbikT)nbikT + t). 
Summing both sides of the previous inequality over t = {l,...,r— 1}, we get 

T-l 

QbikT + Tf - QbikT + if ^ (T^ - l)//^^^^ - 2QbikT) ^ A*fc(fcT + t). (29) 

Using (j27p and the above inequality, we can write 
QbikT + Tf - QbikTf ^ Ab[kf + T^L. + 2Qb(fcT)(A[fc] - ^f,[fc]), (30) 

where we have used the fact that iJbikTf ^ ifnax ^^'^ Mb[^] — Y^J^o fibikT + t). 

Consider the quadratic Lyapunov function L($[fc]) = J2neAf^n[kf ■ We 
now define the one-frame conditional Lyapunov drift as 

A(*[fc]) = E{L(*[fc + 1]) - L(*[fc])|*[fc]}. 

Using ([25)1 and (1501) . we can express the drift for the QMW policy as 

A(*[ft]) < E j ^ (A.[fc]2(i - a,)2 + ^,[fc]2) + ^ (^[fc]2 + rVL.) 



+ J2iPl + ^,^[kr 



geg 



I + 2 Ei ^ (gb(fcT)(Afc[fc] - i,b[k])) 



+ J2 iYr[k]iA,[k]il - ar) - tlr[k\)) + ^ iZg[k]iP, - /i,[fc])) 



beB 



(31) 



17 



Now, we can find a B < oo such that 



since Ej^^JA:]} < oo and E{Xj^j(<)} < co for each n G A/" and channel j. Also, 



{A„[A;]} is i.i.d. and E{^„[0]} = A„. So, we can rewrite (l5lT) as 
A(*[A:]) < B + 2 e| E (i;[fc](A,(l - a,) - /..[A;])) + {Zg[k]{Pg - iig[k])) 



Rearranging the terms and using the definition (jl2p . we get 

A(*[fc]) < B + 2 eJ E Yr[k]K{l -ar) + Y, ^gWa + Qb{kT)\t 

I rGK gee 6e6 



2Ei E$„[fc]/inW 
I nGAA 



(32) 
(33) 



From the definition of the QMW algorithm, it can be easily shown that 



Ei Y ^n[k]^Jin[ 



*[A:],S(fcr),...,S(fcT + r-l)| 
*[fc],S(/cT),...,S(/cT + T- 1) 



where fJ-'„ [k] corresponds to any other stationary randomized policy. Taking 
expectation w.r.t. the channel-estimates in the frame, we get 



Ei Y ^n[k]tin[k] 



*[fc] Ue Y '^^[kVAk] 

) I nej\r 



Thus, using the above fact, we can rewrite (|32|) as 



A($[fc]) <B + 2eJ Y^r[k]Xr{l ~ ar) + YZMPa + Y^b{kT)\t 
I reiz gee bee 



2Ei Y '^^\ky^\k] 



(34) 
(35) 
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Rearranging the terms again and using definition (jl2p . we then get 
A(*[fc]) ^B + 2J2 Yr[k]E{{\r{l - ar) - fi'^[k])\^[k]} 

geg beB 

Now, if the system is stabilizable, i.e. if A e int[A], then a stationary random- 
ized poHcy exists that stabilizes the system. Besides, this policy makes decisions 
independent of the queue-lengths. Taking the fJ.'n[k] decisions to be the service 
decisions corresponding to this stationary randomized policy, we have 

E{A^(1 - ar) - < -e, 

E{/3,-/i;[fc]|*[A;]}<-e, 
E{Xh - ^',[k]\^[k]} < -e, 

where e > 0. Thus, the previous drift inequality can the simplified as 

A(*[fc]) ^B-2e^ $„[fc], 

where we have used definition (jl2p . Now, as before, the queue evolves as a 
Markov chain. The drift inequality above gives negative drift but for a finite set 
of queue-lengths. Hence, using Theorem 2 from [TH], the network is stable. 

Appendix F 

Proof of Lemma 1: 

We are given the Lyapunov function L{Q{t)) = Ylif=i Qii^)- Squaring both 
sides of equation ([Ij, and using the fact that {max{a,0})^ ^ a^, we have 

QUt + 1) - QKt) ^ f^lit) + AUt) + 2Q,{t){Mt) - fi,{t)). 
Summing over all i and taking E{- | Q{t)}, 

N 

A{Q{t)) ^ E{U{t) I Q{t)} + 2Y,Q^{tKK - ntM{t) I Q{t)}) (36) 

i=l 

where U{t) is as defined in ©. Since, E{Af(i)} < oo and E{Xf^{t)} < oo for 
each i and j, 

N 

B^Y. nu{t)\Q{t)] 

i=l 

is bounded and independent of Q{t). Hence, we are done. 
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